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ABSTRACT 
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Given  a  feasible  point  for  a  nonlinear  programming  problem,  we 
investigate  the  structure  of  the  feasible  set  near  that  point.  Under  the 

,  ^  )  A  tj  u- 

constraint  qualification  called  regularity,  we  -show-  how  to  compute  the  tangent 
cone  to  the  feasible  set,  and  to  produce  feasible  arcs  with  prescribed  first 
and  second  derivatives.  In  order  to  carry  out  these  constructions , > we  show 

that  ^particular  way  of  representing  the  feasible  set  (as  a  system  of 

0~ -I V*  ’  ”  17 

equations  with  constrained  variables)  is  particularly  useful.  We  alee  give 
fairly  short  proofs  of  the  first-order  and  second-order  necessary  optimality 
conditions  in  very  general  forms,  using  the  arc  constructions  mentioned  above. 
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SIGNIFICANCE  AND  EXPLANATION 


Nonlinear  programming  is  concerned  with  the  problem  of  minimising  a 
function,  often  fairly  smooth,  over  a  set  (the  so-called  feasible  set) 
described  by  nonlinear  inequality  and  equality  constraints,  as  well  as  perhaps 
some  bounds  or  other  restrictions  on  the  variables*  Problems  of  nonlinear 
programming  arise  in  statistics,  in  chemical  engineering,  in  economics,  and  in 
many  other  areas. 

Certain  basic  conditions,  called  optimality  conditions,  must  be  satisfied 
by  any  candidate  for  a  solution  of  such  a  problem,  provided  that  the 
constraints  satisfy  a  reasonable  regularity  condition.  These  conditions 
describe  the  relationship  of  the  derivatives  of  the  function  being  minimised 
to  the  derivatives  of  the  constraint  functions  and  the  set  over  which  the 
minimisation  is  being  done.  They  form  the  basis  for  most  numerical  algorithms 

for  solving  such  problems.  -  ^ 

v 

This  paper  examines  the  structure  of  the  feasible  set,  and  introduces 


some  effective  and  fairly  simple  ways  of  dealing  with  this  set.  As  a  by¬ 


product  of  these  techniques,  comparatively  simple  and  straightforward  proofs 
of  the  optimality  conditions  and  of  related  results  are  given. 


* 
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LOCAL  STRUCTURE  OF  FEASIBLE  SETS  IN  NONLINEAR  PROGRAMMING,  PART  Is  REGULARITY 

•* 

Stephen  M.  Robinson 


1.  Introductions  the  regularity  condition. 

This  paper  deals  with  solution  sets  of  systems 

h(x)  -  0 


x  e  c  , 

where  h  is  a  Cr  function  (r  £  1)  from  an  open  set  11  in  l"  to  I*,  and  where  C 
is  a  convex  set,  not  necessarily  closed,  in  R° •  He  shall  denote  the  set  of  all  solutions 
of  (1.1)  by  C  n  h" 1 ( 0 ) ,  or  briefly  by  F>  One  of  the  main  application  areas  in  which 
sets  like  F  arise  is  nonlinear  programing,  since  the  constraints  of  many  nonlinear 
programing  problems  either  look  like  (1.1)  or  can  be  made  to  look  that  way  by  simple 
manipulations  such  as  adding  slack  variables.  Therefore,  oust  of  our  analysis  of  (1.1) 
will  be  aimed  at  establishing  results  useful  in  nonlinear  programing. 

Given  a  point  Xg  e  F,  we  often  want  to  know  what  that  part  of  F  near  xQ  is 
like.  However,  it  is  hard  to  tell  much  about  F  by  looking  directly  at  the  nonlinear 
system  (1.1).  Therefore,  a  natural  idea  is  to  simplify  (1.1)  by  linearising  h  about 
Xg  and  then  to  consider  the  system 

h(Xg)  +  h'(Xg)(X  -  Xg)  -  0 
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Since  h(xg)  ■  0  by  assumption,  the  eet  of  solutions  of  (1.2)  is  just 

Xg  +  (C  -  xg)  n  ker  h'(xg),  where  ker  h'(Xg)  denotes  the  set  {s|h'(Xg)z  -  0).  He  shell 
denote  this  solution  eet  by  lF.  Note  thst  lF  is  a  convex  set,  so  we  can  deal  with  it 
considerably  more  easily  than  we  can  with  F. 

An  Implicit  assumption  behind  the  construction  and  use  of  lF  is  that  near  x,  lF  is 
"a  good  approximation”  of  Ft  one  thinks,  for  example,  of  the  tangent  space  to  a  smooth 
manifold.  However,  it  can  easily  happen  that  lF  is  nothing  like  F.  Consider,  for 
enable,  the  case  with  n  •  2  and  m  ■  1 ,  in  which 

h(C,n)  «■  -  n 

and 

c  i«  {(5,n)ln  <  o)  . 

With  these  definitions,  it  is  clear  that  F  is  the  origin  in  It2,  while  if  we  take  Xg 

to  be  the  origin  then  lF  ■  It  *  {0}  ■  {(1,0)  |X  «  ll}.  On  the  other  hand,  if  we  take  the 

same  function  h  but  change  C  to  {({,n)l  C*  -  D  <  0),  then  F  becomes 

{(C,n) I £2  -  n  *  0}  while  lF  becomes  the  origin.  Hence  in  these  two  cases  F  and  lF 

are  not  at  all  alike,  and  in  general  we  will  need  same  criterion  to  ensure  that  lF  is 

locally  similar  to  F  if  we  are  to  use  lF  to  draw  conclusions  about  F. 

The  criterion  we  shall  use  is  an  extension  of  the  familiar  requirement,  for  systems  of 

equations,  that  the  derivative  of  h  at  Xg  carry  ft"  onto  The  extension  consists 

in  taking  appropriate  account  of  the  presence  of  the  convex  set  C,  and  to  do  this  we  need 

the  idea  of  a  tangent  cone.  Since  we  shall  use  this  idea  later  too,  we  introduce  it  in  a 

fairly  general  formi  if  S  is  a  subset  of  ft0  and  x  €  cl  S,  then  the  (Bouligand) 

tangent  cone  to  8  at  x  is  the  set  Tg(x)  consisting  of  all  right  derivatives  of  arcs 

emanating  from  x  with  the  property  that  avery  neighborhood  of  x  meets  the  intersection 

of  S  with  the  arc.  It  is  easy  to  show  that  a  point  d  belongs  to  Tg(x)  if  and  only  if 

there  are  sequences  (s  }  C  s,  converging  to  x,  and  {1  )  C  ((>,+•),  such  that 

n  n 

1  (s  -  x)  converges  to  d.  See,  e.g.,  [2]  for  more  information  about  tangent  cones, 
n  n 

Having  the  tangent  cone,  we  can  now  make  the  extension  we  mentioned  earlier,  by 
defining  regular  points  for  (1.1). 


%. 


DEFINITION i  Let  «„  (  C  with  h(xg)  -  0.  Then  xQ  is  a 
h'(xg)  carries  Tc(Xg)  onto  tP. 


int  of  (1.1)  if 


It  will  be  helpful  to  have  some  equivalent  forms  of  regularity  available.  Three  are 
given  in  the  next  proposition. 

PROPOSITION  1.1 i  Let  h  and  C  be  as  previously  defined,  with  Kg  (  F.  Then  the 
following  are  equivalent! 

(a)  h'  (xQ)  [Tc ( Xq )  1  -  1^  (1.3) 

(b)  h*  (Xq)  (aff  C)  -  ^  and  ker  h*(Xg)  n  ri(C  -  Xg)  *  fi  .  (1.4) 

where  ri  denotes  relative  interior  (see  [4]  for  definitions). 

(c)  0  «  int{h(xQ)  ♦  h' (Xq)(C  -  x0>}  ,  (1.5) 

where  int  denotes  interior. 

Part  (c)  of  Proposition  1.1  is  the  fore  in  which  regularity  was  introduced  in  (3); 
pert  (b)  is  a  generalised  version  of  the  well  known  Mangasarian-Froanvits  constraint 
qualification  [1],  and  part  (a)  is  used  in  (2).  For  our  constructions  in  this  paper,  we 
shall  depend  primarily  on  (b). 

PROOFt  Denote  h’ (Xg)  by  Di  let  B  be  the  unit  ball  of  bP  and  denote  its 
intersection  with  aff(C  -  Xg)  by  Bc. 

(a  — >  b) i  We  know  Tc<Xg)  -  cl  cone(C  -  Xg)  since  C  is  convex*  in  particular 
Tg(Xg)  C  aff(C  -  Xg)  so  that  D(aff  C)  ■  l^1.  If  (ker  D)  n  rl(C  -  Xg)  is  tspty,  then  the 
principal  separation  theorem  [4,  A.  11.3]  guarantees  the  existence  of  a  nonsero  w  «  wT 
such  that  for  each  c  C  C,  <wD,c  -  Xg  >  £  0.  But  then  for  any  s  e  cl  cone(C  -  Xg)  -  Tc(xg) 
we  have  (w,Ds>  *  <wD,s)  £  0,  so  that  D(Tc(xg)]  *  if,  contradicting  (a).  It  follows 
that  (ker  D) n  ri(C  -  Xg)  *  4,  which  proves  (b). 

(b  ~>  c)i  If  D(aff  C)  ■  rf1  then  we  know  D  is  an  open  napping,  so  there  is  a 
bounded  neighborhood  N  of  the  origin  in  aff(C  -  Xg)  such  that  DN  is  a  neighborhood  of 
the  origin  in  rf1.  Let  y  €  (ker  D)  n  ri(C  -  Xg)t  then  for  some  positive 
c,  *g  +  y  ♦  CBc  c  C.  Choose  a  positive  8  small  enough  that  4N  c  *»c»  then 
Xg  ♦  y  ♦  <N  C  c  and  thus  y  ♦  6n  C  c  -  Xg.  But  D[y  ♦  8n]  -  4DN,  a  neighborhood  of  the 


-3- 


origin,  and  thoroforo  D(C  -  Xg )  is  also  a  neighborhood  of  the  origin,  he  f(xg)  -  0, 
this  proves  (e). 

(c  — >  a)  i  As  Tc(*j)  3  C  -  Xg,  (c)  implies  that  D[Tg(Xg)]  is  a  neighborhood  of 
the  origin.  But  it  is  also  a  const  hence  it  must  be  .  This  completes  the  proof. 

With  the  idea  of  a  tangent  cone  and  the  criterion  of  regularity,  we  are  in  a  position 
to  investigate  the  relationships  between  if  and  F.  In  Section  2  we  introduce  a  special 
coordinate  system  that  is  particularly  well  suited  for  this  investigation)  then  in  Section 
3  we  apply  this  construction  to  show  that  F  and  LF  have  the  same  tangent  cone  at  Xg, 
and  to  show  further  that  feasible  arcs  can  be  constructed  with  prescribed  tangents  in  LF. 
Finally,  we  apply  these  arcs  to  give  simple  proofs  of  necessary  conditions  for  optimization 


Construction  of  a  coordinate  system;  feasible  arcs 


In  this  section  we  show  how  to  set  up  a  special  coordinate  system  that  is  fitted  to 
the  structure  of  F  near  a  regular  point  Xg .  Denote  aff  C  by  A,  and  the  subspace 
parallel  to  A  by  Mt  then  we  shall  first  use  the  implicit-function  theorem  to  identify 
that  part  of  A  n  h~ 1  ( 0 )  near  x^  with  that  part  of  M  n  leer  h'  (xQ)  near  0,  under  the 
regularity  hypothesis*  Then  we  shall  show  how  this  identification  leads  to  a  very  simple 
way  of  constructing  feasible  arcs  with  prescribed  derivatives.  These  arcs  will  be  applied 
in  the  next  section  to  prove  results  about  tangent  cones  and  optimality  conditions. 

First,  we  are  going  to  decompose  Rn  in  a  way  that  employs  the  subspaces,  M  and 

ker  h'(Xg),  of  special  importance  to  us.  Let  us  denote  hMxg)  by  D.  Then  since 

D(M)  “  D(aff  C)  «*  if  by  part  (b)  of  Proposition  1.1,  we  must  have 

{0}  -  [D(M)}A  -  {u|D*u  e  M1}  . 

But  then  also  (im  D*)  n  mA  -  {0},  and  taking  orthogonal  complements  we  have 
(ker  D)  ♦  M  ■  Jr".  Denote  (ker  D)  fi  M  by  K,  and  let  J  and  L  be  subspaces 
complementary  to  K  in  ker  D  and  M  respectively «  then  we  have 

if  •  J  •  K  •  L,  J  •  K  -  ker  D,  K  •  L  -  H  .  (2.1) 

Given  the  decomposition  (2.1),  we  denote  by  Pj,  PR  and  P^  the  projectors  from 
■P  onto  the  subspaces  J,  K,  and  L,  in  each  case  along  the  other  two  spaces.  Thus 
Pj  ♦  PK  +  Pj.  ■  I,  and  the  product  of  any  two  of  these  projectors  is  the  sero  operator.  W 

shall  often  write  PQ  for  Pj  ♦  PR,  the  projector  on  ker  D  along  L. 

He  now  construct  a  particular  generalised  inverse  of  D  that  will  aid  us  in  applying 
the  implicit-function  theorem.  Dote  first  that  from  (2.1)  we  have  B11  -  (ker  D)  •  L,  and 


D(L)  -  D(L  e  K)  -  D(M)  »  if  . 

It  follows  that  D  is  a  bijection  from  L  onto  if,  and  the  generalised  Inverse  that  we 
want  is  just  the  inverse  of  this  bijection.  This  will  be  a  linear  operator  D~  from  if 
to  tf1  having  the  properties  that  DD~  is  the  identity  of  if,  and  D-D  is  the 
projector  P^.  To  construct  it,  we  can  choose  any  bijection  E  from  if  to  L  and  let 
D~  i“  KD«)-1 .  The  existence  of  ( DE ) “ 1  is  guaranteed  since  L  is  independent  of 


note 


ker  D,  end  it  is  clear  that  DD-  is  the  identity  of  rf*.  To  show  that  D~D  -  PL, 

that  D~D  annihilates  ker  D,  while  if  1<L,  then  i  “  Ev  for  some  v  e  ■?*,  and  then 

D~Di  -  D— DBV  -  E(DE)"1 (DE)v  -  Ev  -  t  . 

As  bP  «  (ker  D)  •  L  we  must  have  D~D  -  PL,  and  it  follows  from  this  that 
I  -  D-D  -  P0. 

The  construction  of  these  subspaces  and  projectors  nay  be  easier  to  understand  if  we 
relate  the  present  situation  to  the  well  known  case  of  linear  programing.  There,  we  have 

h(x)  -  Ax  -  a,  C  *  h'(x„)  *  A  , 

where  A  is  a  linear  operator  froa  «"  to  hF  and  a  «  The  regularity  hypothesis 
that  we  are  using  implies  in  particular  that  A  has  full  row  rank,  and  there  is  no  loss  of 
generality  in  assuming  that  A  is  partitioned  as  [B  N] ,  with  B  n  x  n  and 
nonsingular.  Then  the  space  of  the  first  m  components  of  rf1  is  clearly  of  dimension 
a  and  independent  of  ker  A,  and  we  shall  take  it  to  be  L.  Now  for  the  bijection  E  we 

can  choose  ^mj ,  where  in  denotes  the  a  x  a  identity  matrix,  and  if  we  write  D  in 

place  of  A  we  have 


D  -  E(DE) 


«  ■  Cl-"  ■  LI 


Pt  -  D 


B-’l  fx  B-’N 

“D  -  [BN]-  , 

.0  lo  0  . 


P0  ■  1  -  PL  - 


The  reader  will  recognise  the  last  n  -  ra  columns  of  PQ  as  being  the  edges  of  the 
feasible  region  along  which  the  simplex  method  can  move  away  froa  xfl. 


Returning  to  the  nonlinear  situation,  let  us  define  e  function  P  i  0  »  in  ♦  *f  by 
P(x,y)  i-  o”h(x)  +  (I  -  D~D)(x  -  (Xg  +  y)l  .  (2.2) 

As  i«  D~  -  L  end  Z  -  D~D  -  Pq ,  the  two  major  terms  on  the  right-hand  side  of  (2.2)  lie 
in  the  independent  spaces  L  and  her  D.  Accordingly,  one  has  P(x,y)  -  0  if  and  only  if 

(i)  D"h(x)  -  0  (that  is,  h(x)  -  0,  since  her  D"  -  (0)),  and 

(ii)  P0[x  -  (Xg  y)]  -  0  (that  is,  x  -  (Xg  +  y)  e  L). 

Of  course,  if  y  €  K  then  (ii)  Implies  that  Pg(x  -  Xg)  -  y,  so  x  -  Xg  +  y  +  A  for  some 
i  e  L.  Thus,  for  x  €  Q  and  y  e  K,  F(x,y)  -  0  if  and  only  if  h(x)  ■  0  and 
x  e  Xg  y  ♦  L. 

If  we  differentiate  (2.2)  with  respect  to  x,  we  find  that  Px(Xg,y)  -  I  for  any 
y  e  tP.  Thus,  we  may  apply  the  implicit-function  theorem  to  produce  neighborhoods  D  of 
the  origin  in  rf1  and  V  of  Xg,  and  a  unique  C*"  function  x  t  0  ♦  V,  such  that 
x(0)  «  xg  and,  for  each  y  «  0,  F(x(y),y)  -  0.  This,  in  turn,  meant  that  for  each 
m  «  U  n  M,  h(x(m)]  “  0  and  Pg(x(m)  -  Xg)  -  P0m  -  P^m.  To  find  the  derivatives  of  this 
function  x,  we  note  that  since  F(x(y),y)  -  0  we  have 

°  -  P(x(y),y)|y.g  -  FJt(x0,0)x,(0)  ♦  Fy(xg,0)  -  x*(0)  -  Pg, 

where  we  have  used  the  fact  that  Fx(Xg,0)  -  I.  Therefore  x'(0)  -  Pg.  Mow,  if  rj2  we 
cen  differentiate  again  to  obtain 


0  -  F(x(y),y)|y-0  -  ^  [F„(x(y)  ,y)x'  (y)  +  Fy(x(y)  ,y)]  ly.g 

-  FjHjtXg'O)*' (O)x'(O)  +  FjjylXg.OJxMO)  ♦  Fx(Xg,0)x-(0) 

♦  Fyx(*0'0,X'<0)  +  *yy<*0'0>  *  (2,3) 

However,  F_,,  F„_,  end  F  are  all  sero  since  the  only  appearance  of  y  in  (2.2)  is  in 

a  linear  termi  also,  we  know  Fx(Xg,0)  ■  I,  “  D_h*(xg),  and  x’(0)  -  Pg. 

Therefore  (2.3)  yields  for  arbitrary  s,ti 

x"(0)(s)(t)  -  -o"ha(Xg)(P0s)(Pgt)  .  (2.4) 

These  derivatives  can  be  further  simplified  if  their  arguments  rasutin  in  ker  D,  since 


there  Pn  acts  like  the  identity. 


The  following  theorem  describes  various  properties  of  x,  some  of  which  we  have  noted 


informally  above. 

THEOREM  2.1:  Suppose  Xq  is  a  regular  point  of  (1.1)  and  let  D,  J,  K,  L  and  M  be 
as  previously  defined.  Then  there  exist  neighborhoods  U  o£  0  _in  r"  and  V  of  Xg 
in  and  a  unique  C*-  function  x  :  U  ♦  V  such  that  x(0)  »  Xg  and,  for  each  y  e  U( 

h[x(y)l  “  0  and  Pgtx(y)  -  (Xg  +  y) J  -  0.  One  has  x' (0)  -  PQ  and,  if  r  £  2, 
x"(0)(s)(t)  »  -D_h"(Xg ) (PgS) (Pjjt)  for  s,t  €  tP. 

Note  that  Theorem  2.1  yields  Pg[x(y)  -  Xq]  «  Pgy,  so  that  if  y  e  K  then 

“  *0!  “  y«  80  that  *<y>  e  «o  ♦  P0[x(y)  -  xoI  +  pt^x(y>  _xo^  exo  +  y  +  LCxo 

+  M  “  aff  C.  Thus  x  maps  portions  of  K  near  0  to  portions  of  (aff  c)  n  h-1 (0 ) 

near  Xg.  However,  we  are  really  interested  in  F  »  c  n  h-1 (0 ) ,  so  we  shall  next  examine 
how  to  keep  x(y)  in  C  instead  of  just  in  aff  C.  Xt  turns  out  that  a  very  easy  way  to 
do  this  is  to  investigate  arcs  in  F.  Our  initial  result  involves  only  first  derivatives. 

THEOREM  2.2:  Let  h  be  C1  and  let  Xg  be  a  regular  point  for  (1.1).  Let 

d  e  rl  Tc(xg).  In  order  that  there  exist  a  c1  arc  w(t)  in  F*  C  rn‘’(o)  with 
w(0)  -  Xg  and  w(0)  «  d,  it  is  necessary  and  sufficient  that  d  e  ker  h'(Xg). 

PROOF  (necessity):  For  all  small  t  we  must  have  h[w(t)]  >0;  thus 

0  “  dt  U-0  “  h*  txo>wl°>  “  h* (Xg)d  . 

(sufficiency):  If  d  «  ker  h'(Xg),  then  since  d  €  ri  Tc(Xg)  C  m,  we  have  d  e  K. 
Also,  since 

d  «  ri  Tc(Xg)  -  ri  cl  cone(C  -  Xg)  -  ri  cone(C  -  Xg)  »  cone  ri(C  -  Xg)  , 
there  exist  11  >  0  and  e  >  0  with  (id  e  rl(C  -  Xg)  and  pd  +  eb^C  c-  xQ,  where 
B||  *  B  ^  M,  the  unit  ball  in  M,  and  where  cone  S  denotes  the  cone  generated  by  the 
set  S.  Now  let  y(t)  1“  td|  then  with  w  :■  x  o  y  we  have,  for  small  non-negative  t, 

w(t)  *  Xg  +  w(0)t  o(t)  .  (2.5) 

But 

w(0)  -  xM0)y(0)  -  Pgd  -  d  . 

Also,  since  y(t)  remains  in  K  we  have  hlw(t))  »  0  and  w(t)  e  aff  C  by  Theorem  2.1; 
hence  w(t)  •  *g  «  M  and,  since  d  «  K  (CM),  also  the  function  denoted  by  o(t)  in 


(2.5)  remains  in  M.  Thus 

w(t)  -  Xq  »  [1  -  U  ’tlO  +  u  1 1 [ »a  +  ut  ^  o( t ) ]  , 
and  for  small  non-negative  t  this  is  a  convex  combination  of  points  of  C  -  x_,  hence 
itself  a  point  of  C  -  xQ  by  convexity.  Thus,  for  all  small  non-negative  t,  w(t)  e  C, 
and  this  completes  the  proof. 

If  we  are  willing  to  assume  that  h  is  C2  instead  of  only  C1 ,  we  can  obtain  a 

2 

C  arc,  and  we  can  prescribe  not  only  its  first  derivative  but  also  its  second. 

THEOREM  2.3s  Let  h  be  C2  and  let  Xg  be  a  regular  point  for  (1.1).  Let 
d  e  C  -  Xq  and  s  6  ri  Tc(Xg).  In  order  that  there  exist  a  C2  arc  w(t)  _in 
F  “  C  n  h  ^(0)  with  v(0)  “  Xq,  w(0)  ”  d,  and  0(0)  »  s,  it  is  necessary  and  sufficient 
that  d  e  ker  h' (Xq)  and 

h" ( Xq ) dd  +  h ' ( Xq ) 8  -  0  .  (2.6) 

PROOF  (necessity)!  Theorem  2.2  tells  us  that  we  must  have  d  e  ker  h'(xQ).  For 

(2.6) ,  recall  that  h[w(t)]  ”  0  for  small  non-negative  t,  so 

d2 

0  -  —j  h[w(t)]  lt.0  -  h“(xQ)dd  +  h’  (xq)s  , 
dt 

where  we  have  used  the  facts  that  w(0)  -  d  and  0(0)  »  s. 

( sufficiency )s  Suppose  d  e  ker  h* (x0)  and  (2.6)  holds.  For  small  non-negative  t 

let 

y(t)  s“  td  ♦  "j  t2PgS  . 

Note  that  s  e  ri  Tc(x0) C  M  >  X  •  L,  so  PQs  6  X.  But 

d  «  (c  -  Xg) n  ker  h’ (Xg)  C  (X  «  L)  n  <j  e  K)  -  X,  so  in  fact  y(t)  «  K.  Let 
w  !«  x  o  y.  for  small  non-negative  t  we  have  h[w(t)J  -  n  and  w(t)  e  aff  C  by  Theorem 
2.1.  Also, 

w(0)  -  x' (0)y(0)  -  P0d  -  d  (since  d  «  X)  , 


»<0)  -  x“(0)y(0)y(0)  ♦  x'<0)?{0) 

-  -D~h"(Xg)dd  +  Pjjs 

-  -D_t-h*  (Xg)sl  +  PQs 

-  <Pt  ♦  Pg)s  -  •  , 

where  we  have  used  (2.6),  Pg  -  pQ,  D~D  -  PL,  and  PL  +  P„  -  I.  Hence  w  has  the 
prescribed  derivatives. 

Now  from  Taylor's  theorea  we  have  for  small  non-negative  t, 

w(t)  «  Xq  ♦  td  ♦  -  t2s  ♦  r(t)  ,  (2.7) 

where  r(t)  -  o(t2).  Note  that  since  w(t)  C  aff  C  and  since  d  e  K(C  M)  and 
s  «  ri  tc(xq)  C  m,  we  sust  have  r(t)  «  M  also,  since 

s  e  ri  Tc(x0)  -  ri  cone(C  -  Xg)  -  cone  ri(C  -  Xg)  , 
there  exist  |i  >  0  and  e  >  0  with 

US  ♦  6Bm  C  C  -  Xg  , 

where,  as  before,  ^  -  B  n  M.  Rewriting  (2.7)  as 

w(t)  -  Xg  -  [1  -  t  -  u_1t2)0  ♦  td  ♦  j  u_1t2tu*  ♦  2ut"2r(t)l  ,  (2.8) 

we  see  that  for  small  non-negative  t  the  right-hand  side  of  (2.8)  is  a  convex  combination 
of  points  of  c  -  Xg,  and  therefore  w(t)  «  C.  But  we  know  h[w(t)]  -  0,  so  w(t)  is 
feasible,  as  required.  This  proves  Theorem  2.3. 

In  this  section  we  have  shown  how  to  use  regularity  to  gain  substantial  amounts  of 
information  about  the  structure  of  F  near  a  regular  point  Xg.  In  the  final  section  we 
show  how  to  use  this  information  to  compute  the  tangent  cone  to  F  at  Xg  and  to 
establish  necessary  conditions  for  optimization  on  F. 


.  Applications i  the  tangent  cone  to  F 


lmal tty  conditions 


In  this  section  we  apply  the  construction  of  Section  2  to  compute  Tp(xg)  when  Xg 
is  a  regular  point,  and  to  give  simple  proofs  of  the  general  first-order  and  second-order 
necessary  optimality  conditions  of  nonlinear  programming.  First  we  consider  the  tangent 
cone. 

THEOREM  3.1:  Let  h  be  C1 ,  and  let  Xg  be  a  regular  point  for  (1.1).  Then 
Tp(xg)  -  Tc(xg)  n  her  h'(xg). 

PROOFi  As  Fee,  we  have  Tp(Xg)  C  Tc<Xg).  If  d  e  Tp(xg)  then  there  is  a 

sequence  {x  }  C  F  with  x_  ♦  x^  and,  for  scam  sequence  {1  }  C  (<),+•), 
n  u  u  n 

X  (x  -  x»)  ♦  d.  Por  each  n  we  have 
n  n  u 

0  ”  h(xn)  “  h<Xg)  ♦  h'lijllx,-  Xg )  ♦  0,,*n  "  *g*>  » 
so 

0  -  h*(x  )X  (x  -  x.)  +  CX  lx  -  xl)lx  -  x-l  ’ollx  -  x  I)  . 

O  n  n  0  n  n  T)  n  0  n  0 

Taking  the  limit,  we  find  d  e  ker  h'(xg),  and  thus  Tp(Xg)  c  Tc*  *0*  n  k*r  *»’ *Xg). 

For  the  opposite  inclusion,  observe  that  regularity  implies 

4  *  ri(C  -  xg)  n  ker  h' (xg)  c  cone  rl(C  -  Sg)  n  ker  h* (xg) 

“  ri  cl  cone(C  -  Xg)  n  ker  h'(Xg)  «  rl  Tc(xg)  n  ker  h'(xg)  . 

Let  d  e  ri  Tc(xg)  n  ker  h'(Xg).  Using  Theorem  2.2,  construct  a  C1  arc  w(t)  in  F 

with  w<0)  -  Xg  ard  w(0)  »  d.  Por  large  n,  define  xn  i-  w(n_1).  We  have  xn  «  F, 

and  X-  +  xn.  Also,  with  X  i-  n  we  have 
n  u  n 

X  (x  -  Xn>  ■  nlw(n-1)  -  w(0)J  ■  n(w(0)n_1  ♦  o(n-1))  -  d  ♦  no(n-1)  . 
n  n  u 

Taking  the  limit  we  find  that  Xn(xn  -  Xg)  ■»  d,  so  d(  Tp(xg).  Hence 

ri  Tc(xg)  n  ker  h'(Xg)  C  Tp(xg)  .  (3.1) 

The  left  side  of  (3.1)  is  the  nonempty  intersection  of  two  relatively  open  sets,  hence  is 
the  relative  interior  of  the  intersection  of  their  closures.  Accordingly,  we  have 

rl[Tc(x0)  H  kar  h* (Xg)]  C  Tp<Xg)  , 
and  since  Tp(Xg)  is  closed  we  have 

Tc(Xg)  n  ker  h* (Xg)  C  Tp(Xg)  . 

This  completes  the  proof. 


Theorem  3.1  yields  a  simple  proof  of  the  first-order  necessary  optimality  conditions 


under  the  hypothesis  of  regularity.  To  see  how  to  construct  such  a  proof,  consider  the 
problem  of  minimising  f  on  F,  where  f  is  a  function  from  fi  to  K. 

THEOREM  3.2s  Let  f  and  h  be  C1,  and  assume  Xq  la  a  regular  point  of  (1.1). 

If  Xg  is  a  local  minimi ser  for  f  on  F,  then  there  is  seem  X  e  if  with 

f'(xg)  ♦  hMxQ)*X  e  -Hc(Xq)  . 

Here  the  asterisk  denotes  the  adjoint  operator,  and  Hc(x0)  is  the  normal  cone  to 
C  at  Xq  defined  by 

Hc(Xq)  -  Tc<Xq)"  -  {y  «  if  |Vs  e  Tc(Xq),  <y,s  )  i  0)  . 
proofs  If  d  (  T p( Xq ) ,  then  there  are  sequences  (x^)  C  F  and  {X^}  c  (0,+») 
with  xn  ♦  Xq  and  XR(xn  -  xQ )  ♦  d.  For  all  large  n,  local  optimality  implies  that 

0  $  *<*„>  “  *(Xg)  *  f  <*o,<Kn  “  *0*  +  ,xn  “  x0*>  ' 

so 

0  i  f’(x  )X  (x  -  x  )  ♦  (X  lx  -  x  l)lx  -  x  r'ollx  -  x  I)  . 
u  ft  n  u  n  n  o  n  o  no 

Taking  the  limit  we  find  that  f(XQ)d  £  0,  end  this  shows  that  f '  (Xq)  e  -H^Xq).  klso, 
by  Theorem  3.1  we  have  Tp(Xg)  -  Tc(Xq)  n  ker  h'(xg),  so 

NfCxq)  -  (Tc(xq)  n  ker  h' (Xq)J*  -  cl[Hc(x0)  +  in  h'(x0)*]  . 

However,  as  noted  in  the  proof  of  Theorem  3.1  the  regularity  condition  implies 
[ri  Tc(Xq>]  n  ker  hMxg)  *  6,  so  by  [4,  Cor.  23.8.1]  we  find  that  Nc(xg)  ♦  im  h'(x0)*  is 
closed  (since  the  normal  cone  to  a  closed  convex  cone  at  the  origin  is  the  polar  of  that 
cone).  Hence,  for  some  X  e  if  we  have 


which  completes  the  proof. 


f'(Xg)  +  h'(Xg)  X  e  -Hc(Xq)  , 


We  can  derive  a  second-order  necessary  condition  for  the  problem  just  considered, 
under  the  seme  regularity  hypothesis,  if  we  are  willing  to  assume  that  f  and  h  are 
C2 .  This  is  done  in  the  next  theorem,  whose  proof  is  based  on  an  unpublished  proof  of 
Weinberger  [5]  for  e  somewhat  different  problem. 

THEOREM  3.3t  Let  f  and  h  be  C2,  and  suppose  that  Xq  is  a  regular  point  of 
(1.1).  If  Xq  Is  a  local  minlmiser  for  f  on  F,  then  for  each 
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d  €  (C  -  Xg)  r>  ker  h'(Xg)  with  f'(Xg)d  -  0,  thgrg  exiats  X  «  tP  such  that 

f'(xg)  h*(x0)*X  e  “Hgtxjj) 

and 

f(Xg)dd  ♦  <X,h'(x0)dd>  i  0  . 

We  not*  that  undar  tha  hypothasaa  of  this  theorem,  if  d  «  (C  -  x„)  n  ker  h’(xg)  than 
tha  first  ordar  condition  implies  f'lxgld  £  0.  Hanca  wa  could  have  substituted 
•f'(Xg)d  J  0"  in  tha  statement  of  tha  thoaraa  without  changing  anything. 

PROOFt  Wa  first  show  that  for  aach  s  «  rl  Tc(xg)  tha  system 

h’(Xg)d-0,  h”(xg)dd  ♦  h'(xg)s  -  0 

f'(x0)d-0,  f(xg)dd  ♦  f'(xj)s  <  0  (3.2) 

d  e  C  -  Xg  , 

is  inconsistent.  Indeed,  suppose  that  s  €  ri  Tc(xg)  and  that  (3.2)  is  consistent.  By 
Theorem  2.3  there  is  a  C2  arc  w(t)  in  F  with  w(0)  ■  Xg,  w(0)  ■  d,  and  9(0)  •  s. 
Define  4  i«  f  O  w.  Than  4(0)  “  0,  4(0)  “  f'(Xg)w(0)  “  f'lxgld  ■  0,  and 

4(0)  -  f(x0)w(0)w(0)  «•  f'(x0)«(0) 

-  f*(x0)dd  ♦  f ' (Xq)s  <  0  . 

Hanca  for  small  t  wa  hava  w(t)  €  F  and 

f Iw(t)]  -  4(t)  -  4(0)  ♦  4(0 )t  ♦  i  4(0)t2  ♦  o(t2) 

-  f(Xg)  ♦  t2[£  4(0)  ♦  t”2o(t2>]  <  f(x0)  . 

Since  this  contradicts  tha  assumption  that  Xg  was  a  local  ainimlser  for  f  on  F ,  (3.2) 

must  be  inconsistent. 

Tha  remainder  of  tha  proof  consists  of  a  separation  argument  designed  to  translate  the 
inconsistency  of  (3.2)  into  a  positive  statement  about  the  existence  of  X.  Choose 
d  «  (C  -  Xg)  n  kar  h'(xg)  with  f'(xg)d  -  0,  and  define  a  linear  transformation 
G  i  h"*1  ♦  by 

h’ ( Xg ) s  ♦  oh"(x0)dd 

G(s,0)  »• 

f'(Xg)s  ♦  of’(x0)dd 

Recall  that  rl  G(Tc(Xg)  »  1^1  ■  0{ritTc(Xg)  »  R+]}  •  G{[ri  Tc(xfl)]  x  lint  R^]  ).  For  any 


•  C  ri  Tg(xg)  and  any  0  >  0,  tha  inconsistency  of  0.2)  for  tha  given  d  and  a 
implies  that 


G(s,C>  t  {0}*  «  i#t  ^  -  ritCO}*  *  *_)  . 

Hence, 

ri  G(Tc(xg)  *  *,.]  n  rit{0>"  «*.]>*. 

By  [4,  Th.  11.3]  thara  axiat  XQ  «  if,  and  u,v  e  ft  with  (X^v)  4  0.  aueh  that  for 
aach  a  «  T^Xg ) ,  r<  V  and  4  €  H., 

U0,h'(x0>«  ♦  yh-(x0)dd>  +  vtf(x0)a  ♦  YfCXg)dd}  >  |i  >  v«  .  (3.3) 

Tha  fora  of  (3.3)  ansuras  that  U  2  0  and  0.  Also,  by  choosing  Y  -  0  in  (3.3)  wa 

obtain 

U0<h'(x0)«>  ♦  vf '  (xQ)s  i  0  (3.4) 

for  aach  a  C  Tc(xg),  and  by  chooaing  a  -  0  and  Y  ■  1  wa  have 

<X0,h«(x0)dd>  ♦  Vf-(xg)dd  *  0  .  (3.5) 

It  v  -  0  than  XQ  *  0  and,  from  (3.4),  for  aach  a  «  Tc(Xg)  wa  hava  <Xg,h*(x0)a>  fc  0. 
But  this  means  that  h* (Xg) (Tc(Xg)]  *  ^ ,  which  oontradlcta  tha  regularity  assumption 
(Proposition  1.1).  Hanca  v  #  0,  so  in  fact  V  >  0.  If  wa  dafina  X  i-  Xg/v  than  (3.4) 
yialda  f'(Xg)  ♦  h'(x0>#X  e  -Hc(Xg),  and  (3.5)  yialds 

f-(Xg)dd  ♦  <X,ha(x0)dd>  l  0  . 

This  completes  tha  proof. 

Ona  night  wondar  whether  wa  could  raplaca  C  -  Xg  in  tha  statement  of  Thaoran  3.3 
by  Tc(Xg).  Zn  ganaral,  the  answer  ia  no.  For  exaaple,  consider  f  «  ft2  ♦  ft  given  by 
t[xy,xz)  «-  *2  ~  2  “I*  with  CCH2  defined  by  C  «■  {(x^XjHXj  >  Ix^3^2},  and  let  h 
be  vacuous.  With  Xg  i-  (0,0)  wo  hava  Tc(xg)  ■  |  x  However,  if  we  taka  d  -  (1,0) 

than  we  hava  f'(Xg)d  -  0  and  fa(Xg)dd  *  -I  <  0,  yet  tha  origin  is  a  local  aininisar  of 
f  on  C.  The  problem  hare  is  that  thara  is  no  feasible  C2  arc  oaanating  from  tha  origin 
with  tangent  (1,0).  Of  course,  if  C  ware  polyhedral  than,  near  Xg,  C  -  Xg  would  agree 
with  Tc(Xg),  so  in  that  case  wo  could  taka  d  «  Tc(Xg). 

It  is  also  not  vary  difficult  to  find  examples  to  show  that  ona  cannot  in  ganaral  use 
tha  sane  X  for  every  d  in  Theorem  3.3.  However,  if  tha  X  appearing  in  tha  first 


order  condition  should  happen  to  be  unique  then  Theorem  3.3  would  guarantee  that  that  X 
would  work  for  all  d.  This  uniqueness  is  in  fact  realised  in  an  important  special  case, 
that  of  nondegeneracy.  That  special  case  will  be  treated  in  detail  in  Part  II  of  this 
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Given  a  feasible  point  for  a  nonlinear  programming  problem,  we  investigate 
the  structure  of  the  feasible  set  near  that  point.  Under  the  constraint 
qualification  called  regularity,  we  show  how  to  compute  the  tangent  cone  to  the 
feasible  set,  and  to  produce  feasible  arcs  with  prescribed  first  and  second 
derivatives.  In  order  to  carry  out  these  constructions,  we  show  that  a 
particular  way  of  representing  the  feasible  set  (as  a  system  of  equations  with 
constrained  variables)  is  particularly  useful.  We  also  give  fairly  short  proof J 
of  the  first-order  and  second-order  necessary  optimality  conditions  in  very 
general  forms,  using  the  are  eQn«trUrtinn« 
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